Alternative Strategies for Variable Selection in Linear Regression Models

1. INTRODUCTION 1.1.1. Variable Selection for Incomplete Data sets In statistical practice, many real-life data sets are incomplete for reasons like non-responses or drop-outs. When a data set is incomplete, practitioners frequently resort to a " case-deletion " strategy within which the incomplete cases are excluded from analysis and the complete cases are formed into a reduced rectangular complete-data format. Then traditional variable selection approaches such as stepwise or criterion-based ones can be applied to such a reduced data matrix. The problem with case-deletion is that too much statistical power may be lost and unbiased estimate is obtained only when the missingness mechanism is missing completely at random (MCAR) (Little and Rubin 1987). A more satisfying strategy for handling general missing data problems is via imputation methods, where the missing data are replaced by some plausible values. In order to retain the original " uncertainty " due to missing data, Rubin (1987) developed a strategy named multiple imputation, within which missing data are imputed several times. Then, the multiple imputed complete data sets are analyzed and the multiple parameter estimates are combined into one that incorporates the between-imputation variance, which can be viewed as the measure of the above uncertainty. Using a Markov chain Monte Carlo imputation approach, Schafer (1997) has developed a program to handle incomplete multivariate normal data with unbiased parameter estimation being achieved when missingness mechanism is " ignorable " in the sense defined by Rubin (1976), where missing values are missing at random (MAR) and the parameters of missingness mechanism and the model parameters are distinct. Ignorability is a less stringent case than MCAR and missingness mechanisms of many practical incomplete data can be well approximated by it. For the purpose of variable selection, however, traditional non-Bayesian approaches cannot be easily applied within the framework of multiple imputation. The difficulty lies in how to combine the selection results corresponding to the multiple imputed data sets, because these Non-Bayesian approaches sometimes describe different " best " linear regression models for different imputed data sets. With different predictors included in these " best " models, regression coefficients cannot just be simply averaged since their explanations vary across models.